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1.  BACKGROUND. 


1.1  The  present  AMOSIST  program  had  Its  beginning  in  a  large  scale 
research  effort  initially  undertaken  at  Fort  Belvoir,  VA,  in  1967,  under 
the  name  Project  AMOS  (automated  Military  Outpatient  System) .  The  in¬ 
tent  was  to  develop  a  means  of  alleviating  the  shortage  of  military 
physicians  which  was  projected  to  follow  the  elimination  of  the  draft. 

The  goal  of  that  effort  was  to  have  been  the  development  of  a  system  of 
medical  algorithms  designed  to  treat  patients  suffering  acute  minor  ill¬ 
nesses.  As  initially  envisioned,  these  algorithms  were  to  have  been 
stored  in  a  computer  and  accessed  by  an  enlisted  Army  medical  corpsman 
(an  "AMOSIST")  via  an  interactive  computer  terminal  equipped  with  a 
video  display  unit.  The  physician-supervised  corpsman  would  serve  merely 
as  the  Interface  between  the  computer  and  the  patient,  asking  the  patient 
the  questions  which  appeared  on  a  video  screen  and  punching  in  (on  the 
terminal  keyboard)  the  patient’s  response  to  each  item.  The  integrity 

of  the  algorithm  would  be  safeguarded  by  the  fact  that  each  logic/ 
decision  block  not  be  displayed  until  the  response  to  the  previous 
question  had  been  made.  Hence,  to  the  extent  that  the  operator  could 
correctly  interpret  the  patient's  response  (usually  a  simple  yes/no 
answer)  and  enter  same  into  the  computer,  the  medical  logic  of  the  al¬ 
gorithm  would  be  preserved  and  the  patient  would  obtain  safe  and  effec¬ 
tive  care — to  the  extent  that  the  algorithm  reflected  same.  However, 
for  a  number  of  factors  (not  well  documented  among  available  Project 
AMOS  reports,  but  believed  to  be  related  to  costs  and  other  issues  per¬ 
taining  to  the  procurement  and  utilization  of  computer  hardware) , 

Project  AMOS  was  terminated  early  in  1973  before  being  implemented  as 
a  computer-based  system  of  health  care  delivery.  As  a  result,  the 
existing  algorithms  were  placed  upon  paper  and  incorporated  into  a 
manual  which  was  to  be  employed  by  AMOSISTs  (in  conjunction  with  super¬ 
vision  from  a  physician)  to  guide  them  in  their  efforts  at  diagnosing 
and  treating  acute  minor  illnesses.  Based  upon  the  favorable  results 
of  a  brief  evaluatory  effort  conducted  by  Project  AMOS  personnel  at  three 
sites  during  the  final  months  of  its  exlstance,  this  hard-copy  version 
of  the  program  was  recommended  for  implementation  as  a  health  care 
delivery  system  within  Health  Services  Command. 

1.2  The  present  study  was  initiated  in  response  to  a  request  from 
the  Surgeon  General  of  the  Army.  The  Surgeon  General  had  received 
requests  from  the  US  Navy  and  the  US  Air  Force  for  *  ‘-«rmation  concerning 
the  operation  and  effectiveness  of  the  AMOSIST  Pro*..  At  the  time  of 
that  request  such  information  was  not  available. 

2.  OBJECTIVES. 

2.1  The  objectives  of  the  study  were  to  evaluate  the  AMOSIST 
Program  as  it  was  being  utilized  within  the  US  Army  Medical  Department 
Activities  (USAMEDDAC)  to  determine: 

a.  The  operational  characteristics  of  the  program. 


b.  The  extent  of  program  acceptance. 

c.  The  extent  to  which  a  physician  savings  was  being 

realized. 

d.  The  cost  effectiveness  of  the  program. 

e.  The  safety  and  effectiveness  of  care  provided. 

3.  METHODOLOGY. 

3.1  The  general  approach  was  to  collect  data  relevant  to  the  cited 
objectives  in  two  phases  through  a  combination  of  on-slte  visits  and 
®*iled  surveys.  The  operational  characteristics  of  the  program  were 
determined  from  response  to  a  large  scale  checklist  which  addressed 
the  functions  of  an  AMIC .  Program  acceptance  was  determined  through 
a  series  of  questionnaires  administered  to  both  patients  and  staff 
(Inclusive  of  both  AMIC  and  non-AMIC  hospital  staff).  The  issue  of 
physician  savings  was  addressed  by  determining  the  average  time  re¬ 
quired  to  treat  a  patient  seeking  care  in  an  AMIC  and  comparing  same 
to  the  average  time  required  to  treat  a  patient  seeking  care  in  the 
traditional  counterpart  of  the  AMIC,  the  General  Outpatient  Clinic 
(GOC) .  The  cost  effectiveness  evaluation  was  limited  to  a  direct 
comparison  of  care-provider  cost  (salaried  cost  per  patient  treated  in 
an  AMIC  versus  salaried  cost  per  patient  treated  in  a  GOC).*  The  safety 
with  which  AMOS I ST  delivered  care  was  provided  was  addressed  principally 
by  determining  the  extent  to  which  AMOSISTs  recorded  their  findings  and 
treatment  on  their  data  collection  sheets  in  accordance  with  the  logic 
and  direction  inherent  in  the  algorithms  themselves.  The  effectiveness 
of  the  care  delivered  was  evaluated  by  comparing  AMIC-treated  patients 
to  GOC-treated  patients  as  regards  the  relative  frequency  with  which  they 
made  non-dlrected  returns  to  the  hospital  within  14  days  of  an  initial 
visit  to  obtain  additional  care  for  the  same  illness  or  a  related  illness. 


*The  hourly  wage  equivalent  for  each  category  of  care  provider  was  derived 
from  salary-related  information  provided  in  the  November  1977  Department 
of  the  Army  (DA)  update  (DA  Message:  DACA-FAA-C  0822382N77)  to  Army  Regu¬ 
lation  37-108.  ?uch  calculations  were  based  upon  a  264  day  work  year  at 
eight  hours  per  aay .  (Several  levels  of  physician  hourly  wage  equivalents 
were  computed  based  upon  various  combinations  of  pay  grade  and  special 
pays  and  variable  incentive  pays) .  The  care  provider  cost  associated  with 
the  treatment  of  each  patient  was  then  computed  by  determining  the  amount 
of  time  that  each  category  of  care  provider  spent  in  treating  the  patient 
(to  include  intra-clinic  consultations  by  physician  to  non-physician  care 
providers) ,  multiplying  each  of  these  amounts  of  time  (converted  to 
fractions  of  an  hour)  by  the  hourly  wage  equivalents  appropriate  to  each 
care  provider  involved,  and  summing  these  amounts  for  all  care  providers 
who  had  participated  in  the  patients  care. 


3.2  The  initial  phase  of  the  study  occurred  during  the  months  of 
October,  November,  and  December  1976.  It  involved  both  a  mailed  survey 
to  all  US  Army  Medical  Activities  using  the  AMOSIST  Program,  and  sub¬ 
sequent  on-site  visits  to  twelve  facilities  randomly  selected  from 
those  receiving  the  mailed  survey.  The  mailed  survey  package  included 
patient  satisfaction  questionnaires  to  be  completed  by  patients  seeking 
care  in  the  AMIC,  job  satisfaction  questionnaires  to  be  completed  by 
both  AMIC  and  non-AMIC  hospital  staff,  and  a  large  checklist  which 
addressed  the  functions  of  the  AMIC.  The  checklist  was  to  be  completed 
by  a  non-AMIC  staff  member.  The  on-site  visits  were  of  three  days 
duration  at  each  site  and  were  begun  immediately  after  the  date  at  which 
the  mailed  survey  materials  were  to  have  been  returned.  The  personnel 
performing  these  visits  sought  not  only  patient  and  staff  satisfaction 
Information,  but  also  completed  the  AMIC  checklist  first  hand  and  indivi¬ 
dually  interviewed  the  chiefs  of  clinics  and  services  to  which  the  AMICs 
commonly  referred  patients. 

3.3  The  second  phase  of  the  study  occurred  approximately  six  months 
after  the  initial  phase.  It,  too,  involved  both  mailed  survey  and  on¬ 
site  visits.  The  follow-up  mailed  survey  was  completed  in  April  1977. 

It  consisted  of  the  same  materials  included  in  the  initial  survey.  The 
on-site  visits  during  this  phase  were  of  two  weeks  duration  at  each  of 
three  AMICs  and  three  GOCs.  The  three  AMICs  involved  were  among  those 
AMICs  (of  the  12  visited  during  Phase  I)  whose  operation  was  judged  to 
be  In  greatest  accord  with  existing  program  guidelines  and  evidencing  the 
greatest  apparent  efficiency  and  effectiveness.  The  three  GOCs  selected 
were  those  whose  operation  was  judged  to  be  the  most  traditional;  i.e., 
those  relying  largely  on  physician-provided  care  and  employing  the  least 
number  of  non-physician  personnel  in  direct  care-provider  roles  and  those 
having  the  greatest  workload.  Data  on  patient  satisfaction  were  obtainei 
at  each  site,  and  timed  measurements  were  obtained  for  each  patient  as  he 
progressed  through  the  clinic  and  received  care.  The  principal  temporal 
measures  obtained  were  the  following:  (a)  the  initial  patient  waiting 
time,  (b)  the  time  required  to  obtain  and  record  vital  signs,  (c)  the  time 
required  for  care  to  be  delivered  by  the  patient's  primary  care  provider, 
(d)  the  time  required  for  intra-clinic  consultations  rendered  by  clinic 
physicians  to  non-physician  care  providers  in  the  clinic  (e.g.,  to 
AMOSISTs  and  Nurse  Clinicians/Practitioners) ,  (e)  the  time  required  for 
any  absences  from  the  clinic  due  to  the  need  for  laboratory  tests  and/or 
x-rays,  and  (f)  total  time  in  clinic.  The  times  utilized  to  compute  the 
care-provider- times  used  in  the  cost  effectiveness  and  physician  savings 
evaluations  included  all  time  spent  by  the  patient's  primary  care  provider 
(inclusive  of  any  time  spent  with  the  patient  subsequent  to  his  return 
from  having  laboratory  tests  or  x-rays  completed) ,  all  time  spent  (if 
required)  by  the  primary  care  provider  in  waiting  for  and  obtaining 
intra-clinic  consultations  from  a  physician,  and  (if  performed)  the  time 
spent  by  the  physician  providing  the  requested  consultation.  The  care- 
provider  costs  associated  with  the  treatment  of  each  patient  were 
determined  by  multiplying  each  of  the  aforecited  times  (converted  to 
fractions  of  an  hour)  by  the  hourly  cost  equivalent  of  the  salary  approp¬ 
riate  to  each  category  of  care  provider  involved  and  summing  them. 


4.  FINDINGS. 


4.1  Operational  characteristics. 

4.1.1  General.  The  AMOSIST  program  operational  guidelines  are 
in  two  documents,  AR  40-48,  Health  Care  Extenders,  9  June  1975,  and 
Ambulatory  Patient  Care  Model  #13,  Annex  B,  AMOSIST  Program  Implemen¬ 
tation  Packet  for  Screening  and  Acute  Minor  Illness  Clinic,  US  Army 
Health  Services  Command,  (HSC  APC  Model  #13),  August  1976.  The  Array 
Regulation  specifies  the  requirements  to  Initiate  an  AMOSIST  program. 

The  HSC  APC  Model  #13  provides  a  substantial  set  of  guidelines  for  the 
operation  of  the  program.  These  guidelines  are  for  a  closely  supervised 
program  demanding  strict  adherence  to  the  algorithms. 

4.1.2  An  opinion  was  obtained  from  The  Judge  Advocate  General  In 
1973  regarding  the  AMOSIST  program.  It  was  emphasized  that  the  AMOSIST 
must  be  closely  monitored,  the  history  forms  must  be  completed,  the 
algorithms  meticulously  followed,  and  that  patients  must  be  Informed 
that  the  AMOSIST  is  not  a  physician. 

4.1.3  Most  MEDDACs  had  documented  the  requirements  for  establish¬ 
ing  the  program  as  required  in  AR  40-48  (e.g.,  credentlallng  papers,  and 
a  Therapeutics  Agent  Board  reviewed  and  Commander  approved  drug  list) . 
Most  sites  had  a  suitable  physical  facility  and  work  environment. 

4.1.4  A  significant  number  of  sites  were  deficient  In  one  or  more 
of  the  operational  aspects  discussed  below.  Most  of  these  are  addressed 
by  HSC  APC  Model  #13. 

4. 1.4.1  A  substantial  portion  of  the  sites  have  eliminated  or 
modified  the  triage  portion  of  the  program.  This  Is  an  essential  portion 
of  the  program  intended  to  select  those  patients  who  can  be  treated  by 
the  program's  algorithms. 

4. 1.4.2  Few,  If  any,  sites  have  an  adequate  audit  program.  This 
is  a  major  defect  of  the  operating  programs  and  also  a  shortcoming  of 
the  controlling  guidelines.  A  valid,  rigorous,  and  timely  audit  program 
that  Is  within  the  resources  and  time  constraints  of  the  treatment 
facility  needs  to  be  developed. 

4.1.4. 3  AMOSISTs  are  not  always  constrained  to  follow  the 
algorithms'  logic  in  the  diagnosis  and  treatment  of  the  patient.  Indeed, 
they  are  not  always  limited  to  treating  only  those  diseases  or  symptoms 
complexes  covered  by  the  AMOSIST  Manual. 

4. 1.4. 4  Both  the  guidelines  furnished  and  the  practices  employed 
to  inform  the  patient  of  the  non-physician  status  of  the  AMOSIST  are  of 
questionable  effectiveness. 


4. 1.4. 5  There  is  evidence  of  lack  of  mutual  understanding  between 
AMIC  and  other  clinics  regarding  their  respective  operational  policies 
and  limitations.  The  referral  and  acceptance  of  patients  is  hampered 
by  this  lack  of  understanding. 

4. 1.4.6  The  supervising  physician  must  know  the  entire  AMOSIST 
system  if  he  is  to  be  effective.  Some  sites  do  not  have  a  permanently 
assigned  physician  and  depend  on  a  roster  assigned  supervisor,  while 
others  have  an  individual  assigned  the  responsibility  but  still  have 

a  significant  amount  of  the  physician  staffing  on  a  rotating  basis. 

This  was  found  to  be  extremely  deleterious  to  AMIC  operations. 

4.2.  Program  acceptance. 

4.2.1  Patient  acceptance.  The  findings  showed  that  patients  were 
well  satisfied  with  the  manner  in  which  care  was  delivered  by  the  AMIC 
care  providers.  In  general,  it  was  the  older  patient,  the  retired 
patient,  the  officer  or  his  dependent  patient  who  reported  the  greatest 
satisfaction.  Comparison  of  patient  satisfaction  of  those  treated  by 
an  AMOSIST  with  those  treated  by  a  physician  in  a  General  Outpatient 
Clinic  (GOC)  showed  a  statistically  significant  greater  satisfaction 
with  the  AMOSIST.  This  greater  satisfaction  was  consistent  in  such 
patient  rated  specifics  as  thoroughness  of  examination,  amount  of  informa¬ 
tion  provided  to  patient,  amount  of  interest  shown  in  patient  as  an 
individual,  amount  of  time  spent  with  the  patient,  and  in  courtesy  and 
friendliness . 

4.2.2  AMOSIST. 

4.2.2. 1  Job  satisfaction.  There  was  an  acceptable  level  of  job 
satisfaction  expressed  by  the  AMOSIST.  It  was  of  the  same  level,  over¬ 
all,  as  his  counterpart  working  elsewhere  in  the  hospital.  Those  with 
a  basic  MOS  of  91C  expressed  greater  satisfaction  than  the  91B  AMOSIST 
and  the  non- AMDS 1ST  9 1C. 

4. 2. 2. 2  Program  evaluation.  Again  the  91C,  and  also  the  recently 
trained  91B  AMOSIST,  was  more  secure  and  had  a  higher  opinion  of  the 
program.  The  most  striking  variable  of  the  AMOSISTs'  evaluation  of  the 
program  was  his  perception  of  the  supervisory  physician's  level  of 
knowledge  of  the  program.  Roster  assigned  physicians  were  not  perceived 
to  have  the  knowledge  to  operate  well  within  the  program. 

4.2.3  AMOSIST  Physicians.  The  overall  job  satisfaction  of  AMOSIST 
physicians  was  considered  to  be  satisfactory.  They  expressed  a 
significantly  higher  level  of  satisfaction  than  other  military  physicians 
serving  in  outpatient  clinics  in  the  same  hospital. 

4.2.4  Non-AMIC  Hospital  Staff.  The  acceptance  of  Che  AMOSIST 
program  by  other  hospital  staff  (physicians,  nurses,  enlisted  personnel) 
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was  considered  to  be  moderate.  There  was  marked  variation  in  acceptance 
with  physicians  ratings  the  highest  and  nurses  the  lowest.  Within  any 
group,  the  perception  of  need  for  the  program  and  of  the  adequacy  of 
care  were  highest  in  those  with  the  most  knowledge  of  the  program. 

4.2.5  Difference  in  Program  Acceptance  Between  Phase  I  and  II. 

In  the  six  months  between  the  data  collection  in  Phase  I  and  II  there 
was  a  decrease  in  the  reported  job  satisfaction  of  the  AMOSIST  and  a 
lowering  of  the  AMOSIST  physicians'  evaluation  of  the  AMOSISTs  they 
supervised.  The  significant  program  change  during  this  Interval  had 
been  the  removal  of  the  91C  MOS  from  the  AMOSIST  positions.  Thus  the 
group  that  had  been  the  most  satisfied  with  their  job  was  removed. 

The  average  91C  had  more  hospital  training  (vocational  nurse),  was  usually 
older  and  more  mature,  and  had  more  service  than  the  average  91B  AMOSIST; 
so  it  is  not  unexpected  that  the  supervising  physician  would  consider 
them  more  competent.  The  patients  who  were  seen  by  the  AMOSIST  during 
this  later  evaluation  period  did  not  agree,  they  reported  a  slightly 
higher  level  of  satisfaction  than  before. 

4.3  Physician  Savings. 

4.3.1  The  actual  time  that  a  patient  spent  with  a  physician  at  the 
primary  clinic  was  measured  in  the  AMIC  and  in  the  General  Outpatient 
Clinic  (GOC) .  It  was  significantly  less  in  the  AMIC  (2  minutes  versus 

6 . 5  minutes) . 

4.3.2  This  would  be  a  specious  finding  indeed  if  the  AMIC  patients 
were  referred  to  other  or  secondary  clinics  in  significant  numbers  while 
the  GOC  physicians  disposed  of  their  patients  without  further  consultation. 
To  account  for  this,  the  total  time  a  patient  spent  with  any  physician  in 
the  primary  clinic  or  by  referral  to  a  secondary  clinic  was  calculated. 
Again,  the  AMOSIST  treated  patients  were  found  to  use  less  physician  time 
than  those  treated  at  a  GOC  (five  and  one-half  minutes  versus  nine  minutes) . 

4.4  Cost  effectiveness.  The  cost-per-patient  treated  was  computed 
based  on  care-provider  related  costs.  Consideration  was  given  to  this 
care  cost-per-patient  to  include  referral  to  secondary  clinics.  Several 
levels  of  physician  costs  per  hour  were  calculated,  derived  from  various 
combinations  of  rank  and  physician  special  pay  provisions.  At  all  but 
the  lowest  (and  unrealistic)  levels  of  physician  cost,  the  average  care- 
provider  related  cost  for  the  patient  receiving  AMOSIST  delivered  care, 
was  significantly  lower  than  the  average  for  the  patient  receiving  GOC 
physician  care. 

4.5  Effectiveness  of  AMOSIST  care.  As  measured  by  the  non-dlrected 
return  of  patients  seeking  care  for  the  same  or  related  condition,  there 
was  no  significant  difference  between  AMIC  and  GOC  treated  patients. 


4.6  Safety  of  AMOSIST  care. 


4.6.1  The  evaluation  of  safety  is  measured  by  the  adherence  to  the 
algorithm  by  the  AMOSIST.  This  is  the  basis  of  the  program.  It  is  only 
safe  to  have  an  essentially  untrained  person  diagnosing  and  prescribing 
to  patients  if  he  follows  directions  exactly.  The  directions  in  this 
program  are  to  triage  the  patients  so  the  AMOSIST  treats  only  those  whose 
complaint  is  addressed  by  an  algorithm  and  that  he  then  rigidly  follows 
the  stepwise  logic  of  the  algorithm. 

4.6.2  The  absence  of  the  triage  step  at  some  sites  has  already  been 
mentioned.  This  omission  removed  the  safety  check  that  the  AMOSIST  will 
treat  only  patients  covered  by  the  algorithm. 

4.6.3  Data  Collection  Sheets  (the  sheets  upon  which  AMOSISTs  record 
their  findings  and  treatment)  were  reviewed  to  see  if  the  necessary  en¬ 
tries  were  made,  that  the  algorithms  required,  to  arrive  at  the  diagnosis 
recorded.  Only  single  complaint  records  were  used  so  the  logic  trail 
would  be  clear.*  Any  record  where  the  AMOSIST  asked  for  a  consultation 
with  the  physician  during  the  logic  progression  were  also  excluded  since 
any  deviation  may  have  been  authorized  by  the  medical  officer. 

4.6.4  Among  these  single  diagnosis  Data  Collection  Sheets,  less  than 
3%  were  completed  in  accordance  with  the  algorithm.  This  compares  with  a 
50%  error-free  rate  for  PA  level  care  providers  using  algorithms  (as  re¬ 
ported  in  the  medical  literature)  and  a  70-80%  error  free  rate  for  the 
BAMC  AMIC  (a  closely  audited  program) .  The  error  rate  reflected  in  the 
present  findings  represent,  therefore,  an  unsatisfactory  level  of  compliance. 

4.6.5  Safety  of  a  program  is  enhanced  by  audit  and  correction  action 
based  on  the  audit.  At  no  MEDDAC  visited  was  their  routine  audit  a  tedious 
and  time  consuming  exercise. 

4.6.6  The  non-conformity  to  acceptable  standards  of  safe  operation 

of  this  program  has  resulted,  in  part,  from  inadequate  regulatory  documenta¬ 
tion.  The  program  guidance  in  Army  Regulations  (AR  40-2  and  40-48)  are 
followed  when  they  are  specific  (prescription  practices),  but  are  of  little 
value  when  they  are  general.  For  example,  AR  40-48  indicates  "...  limits 
of  functions  for  AMOSISTs  are  delineated  in  logic  flow  charts  which  are 
physician  prepared."  Specifics  of  details  of  the  program  that  would  result 
in  a  safe  program  are  in  HSC  APC  Model  #13,  a  publication  that  is  not 
directive  but  advisory  in  nature.  Audit  methodology  and  standards,  audit 
feedback,  source  and  validation  of  algorithms,  and  forms  to  be  used  should 
be  published  in  a  directive. 

5.  CONCLUSION. 

The  present  operating  AMOSIST  program  has  been  shown  in  this 
study  to  be  fulfilling  its  intent  in  offering  medical  care  for  acute 
minor  illness.  It  is  cost-effective  in  comparison  to  physician  care  in 
a  general  outpatient  clinic;  it  saves  physician  time;  it  is  well  accepted 
by  patients;  hospital  community  acceptance  varies  from  well  to  tolerance 
in  direct  relationship  to  the  evaluator's  factual  knowledge  of  the  program 

*It  is  noted  that  the  single  diagnosis  constraint  represents  the  "simplest 
case"  condition. 


operation.  This  study  shows  It  to  provide  as  effective  treatment 
as  that  of  physicians  In  general  outpatient  clinics.  Safety  for  the 
patient  depends  on  triage  to  assure  that  there  is  an  algorithm  for  his 
complaint  and  then  on  the  algorithm  being  followed  exactly  to  arrive 
at  a  diagnosis  and  treatment.  The  study  determined  that  neither  of 
these  safety  criteria  are  present.  When  an  AMOS I ST  treats  a  patient 
with  no  algorithm  or  if  he  deviates  from  the  algorithm  logic,  he  Is 
using  medical  Judgement,  a  process  he  is  not  trained  for.  The  safety 
of  the  program  for  the  AMEDD  depends  upon  compliance  to  The  Judge 
Advocate  General's  opinion  on  acceptable  standards  for  legal  protection. 
The  principal  requirements  were  that  the  AMOSIST’s  non-physician  status 
be  made  clear  to  the  patient,  the  algorithms  be  followed  exactly,  and 
that  there  be  close  monitorship  to  assure  adherence  to  the  algorithm 
(audit) .  These  are  not  adequately  controlled  for  the  AMEDD  to  be  safe 
from  criticism  or  sanctions  if  injury  were  to  occur.* 

6.  RECOMMENDATION. 

The  recommendations  of  the  previous  reports  are  appended; 
review  of  the  study  data  indicates  no  changes  in  those  recommendations. 
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*The  records  of  the  Department  of  Legal  Medicine,  firr**  Forces  Institute  of 
Pathology,  show  that  since  the  start  of  the  AMOSIST  program  until  1  January 
1979,  that  of  the  109  Individuals  involved  in  allegations  of  malpractice 
(92  cases)  in  CONUS  Army  medical  treatment  facilities,  only  one  AMOSIST  was 
mentioned . 


5.  CONCLUSIONS 


5.1  Operational  Characteristics.  As  regards  the  operational  charac 
teristics  of  the  AMOSIST  program,  it  is  concluded  that: 


5.1.1  The  documents  required  to  initially  implement  an  AMOSIST 
Program  are  in  existence  at  most  sites. 


5.1.2  The  physical  setting  and  work  environment  is  suitable  at  most 


sites 


5.1.3  An  appreciation  of  the  scope  and  limitations  of  the  care 
provided  in  AMOSIST-staf fed  clinics  is  not  present  to  an  adequate  degree 
in  most  non-AMOSIST  clinics/services. 


5.1.4  The  triage  function  has  not  been  appropriately  implemented  at 
a  substantial  portion  of  sites. 


5.1.5  Patient  disposition  practices  employed  in  AMICs,  and  the 
reactions  to  same  for  non-AMtC  clinics,  reflect  a  lack  of  mutual  under¬ 
standing  on  the  part  of  both  AMICs  and  non-AMIC  clinics  regarding  their 
respective  operational  policies  and  limitations. 


5.1.6  Formally  implemented,  adequate,  regularly  performed,  well 
documented  auditing  programs  are  largely  absent. 


5. 1.6.1  Descriptions  and  guidelines  for  efficiently  performing  audits 
are  not  adequately  provided  in  existing  system-defining  documents  and 
materials. 

5. 1.6. 2  The  guidelines  and  procedures  employed  to  inform  patient  of 
the  non-phyaiclan  status  of  AMOSIST  are  of  questionable  merit  and 
effectiveness. 


5.1.7  AMOSISTs  are  not  closely  constrained  to  the  scope  and  manner 
of  medical  practice  delineated  in  the  algorithms  contained  in  the  AMOSIST 
Manual. 


5.1.8  From  the  vantage  point  of  the  legal  opinion  rendered  by  the 
Judge  Advocate  General's  Office,  the  AMOSIST  program  is  operating  in  a 
manner  which  leaves  it  vulnerable  to  a  successful  suit  by  alleged 
medical  malpractice  resulting  from  AMOSIST-delivered  medical  care. 


5.2  Program  Acceptance.  As  regards  the  issue  of  program  acceptance 
it  is  concluded  that: 


5.2.1  An  acceptable  level  of  patient  satisfaction  exists 


5.2.2  An  acceptable  level  of  job  satisfaction  exists  among  AMOSISTs 
and  AMOSIST  physicians;  l.e.,  the  overall  job  satisfaction  for  both  non¬ 
trainee  AMOSISTs  and  AMOSIST  physicians  equal  that  of  their  counterparts 
who  serve  in  non-AMIC  outpatient  facilities. 


5. 2. 2.1  AMOSIST  trainees  or  those  AMOSISTs  who  have  recently  completed 
their  OJT  phases  of  training  are  more  satisfied  with  their  work  than  are 
their  more  senior  co-workers. 

5. 2. 2. 2  AMOSISTs  having  a  91C  MOS  are  more  satisfied  with  their  work 
than  are  AMOSISTs  with  a  91B  MOS. 

5.2.2.3  Non-trainee  AMOSISTs  working  at  sites  wherein  roster-assigned 
physicians  supply  a  substantial  portion  of  the  physician  man-hours  provided 
to  the  AMIC  report  less  satisfaction  with  their  work  than  do  their  counter¬ 
parts  who  work  at  AMICs  which  are  staffed  primarily  with  physicians  who  are 
regularly  assigned  to  the  clinic. 

5.2.3  Acceptance  of  the  AMOSIST  Program  is  judged  to  be  moderate  by 
medically  trained  personnel  who  work  in  other  clinics/services  which  provide 
outpatient  medical  care;  those  who  report  a  greater  knowledge  of  the  AMOSIST 
Program  (1)  indicate  that  a  greater  need  exists  for  such  a  clinic  and  (2) 
perceive  the  competence  of  the  AMOSIST  to  be  higher. 

6.  RECOMMENDATIONS:  It  is  recommended  that: 

6.1  More  complete  and  definitive,  legally  binding  program  document(s) 
be  written  (i.e.,  either  an  HSC  Supplement  to  AR  40-48,  a  substantial 
revision  and  enlargement  of  AR  40-48,  or  a  new  Army  Regulation)  and  that 
the  provisions  of  the  reconmended  document  be  consistent  with  the  positions 
taken  in  the  legal  opinion  which  has  been  rendered  by  the  Office  of  The 
Judge  Advocate  General. 

6.2  Directives  pertaining  to  the  development  and  approval  of  local 
changes  or  additions  to  the  AMOSIST  Manual  be  more  complete  and  include 
a  requirement  for  HSC  review  and  approval. 

6.3  Auditing  requirements  be  standardized  as  to  type,  frequency,  and 
documentation  and  that  such  requirements  be  accompained  by  fully  explicated, 
efficient  programs  of  application. 

6.4  Command  directives  emphasize  to  local  commanders  the  necessity  to 
involve  Chiefs  of  specialty  clinics  in  the  development  and/or  modification 
of  a  local  AMOSIST  Program  guidelines  pertaining  to  the  referral  of 
patients  from  the  AMIC  and  the  AMIC  Traiage  desk. 
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6.5  Facilities  be  required  to  use  a  regularly  assigned  physician(s) 
to  provide  at  least  four  hours  per  day  in  the  AMIC  fulfilling  the  role 
of  an  AMOSIST  physician. 


6.6  Guidelines  pertaining  to  patient  informedness  be  revised  to 
eliminate  the  need  for  AMOSIST  themselves  to  inform  the  patient  of  this 
own  non-physician  status,  and  that  favorable  consideration  be  given  to 
the  replacement  of  the  term  AMOSIST  (as  applied  to  the  enlisted  care- 
providers  in  the  AMIC)  with  one  which  more  immediately  and  obviously 
identifies  him  as  a  non-physician. 
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5.  CONCLUSION. 

As  regards  the  clinics  included  in  the  present  study,  it  is  concluded 
that: 

5.1  AMIC  and  AMOSIST  delivered  care  requires  less  physician  time  than 
does  GOC  and  GOC  physician  delivered  care. 

5.2  AMIC  and  AMOSIST  delivered  care  is  less  costly  than  GOC  and  GOC 
physician  delivered  care  at  the  levels  of  physician  care/  consultation 
costs  which  are  likely  to  exist  within  most  clinics. 

6.  RECOMMENDATIONS. 

No  recommendations  are  made  as  a  result  of  the  present  findings. 
Concern  has  been  previously  expressed  (Schopper,  1978)  regarding  other 
facets  of  the  AMOSIST  Program  not  herein  addressed,  and  the  data  have  not 
yet  been  analyzed  regarding  the  more  important  issues  of  "safety  and 
effectiveness  of  care."  Final  recommendations  will  appear  in  an  executive 
summary  to  follow. 


APPENDIX  C 

AMOS I ST  PROGRAM  FIELD  EVALUATION:  SAFETY  AND  EFFECTIVENESS  OF  CARE 


5.  SUMMARY  OF  FINDINGS 


5.1  Findings  Related  to  Safety  of  Care  Delivered 

5.1.1  Presence  of  Vital  Signs  on  DCS.  The  vital  signs  were  found  to 
be  relatively  infrequently  recorded.  Among  the  sample  of  282  single 
diagnosis  DCSs  completed  solely  by  AMOSISTs,  only  60.9  percent  has  the 
temperature  recorded,  none  evidenced  a  pulse  rate,  0.7  percent  depicted 
the  respiratory  rate,  and  12.6  percent  showed  the  pateint's  blood  pressure. 

5.1.2  Treatment  DCS  Error  Rates.  Two  hundred  eighty  two  DCSs  AMOSIST 
completed  DCSs  showing  one  diagnosis  and  no  evidence  of  a  physician  consul¬ 
tation  were  examined  to  determine  to  what  extent  the  data  recorded  therein 
was  in  accord  with  the  logic  and  direction  provided  in  the  algorithms. 

For  the  purpose  of  the  study,  two  independent  types  of  errors  were  identi¬ 
fied  which  correspond  to  differences  reflected  on  the  data  entry  points  of 
the  DCSs  themselves.  Type  A  errors  were  those  denoting  the  presence  of  a 
positive  (+)  entry  in  data  entry  points  which  had  been  specially  marked  on 
the  DCS  (with  an  asterisk)  to  inform  the  AMOSIST  that  if  the  symptoms 
corresponding  to  same  existed,  he  must  seek  a  physician  consultation  prior 
to  completing  a  treatment  plan  for  a  patient.  Type  B  errors  were  those 
which  denoted  the  presence  of  entries  to  the  remaining  (non-asterisked) 
data  entry  points  which  were  contrary  to  the  logic  of  the  algorithm. 

These  logic-related  errors  should  also  have  resulted  in  a  consultation  with 
a  physician  since  an  AMOSIST  is  not  permitted  to  assume  the  entire  respon¬ 
sibility  for  the  care  of  the  patient  unless  that  care  is  in  total  accord 
with  the  algorithm.  Both  types  of  errors,  therefore,  reflect  errors  of 
omission  since  both  should  have  resulted  in  a  consultation  with  a  physician- 
an  action  which  did  not  occur  in  the  sample  of  the  DCSs  evaluated.  Independ 
ently  considered,  the  results  of  the  analysis  for  Type  A  errors  indicated 
that  a  physician's  consultation  was  not  rendered  as  required  for  73  percent 
of  the  282  cases  evaluated.  A  separate  analysis  addressing  only  Type  B 
errors  showed  that  an  even  greater  proportion,  90.1  percent,  of  the  DCS 
should  have  reflected  a  physician's  involvement  because  of  some  discrepancy 
between  the  algorithm's  logic  and  the  AMOSIST' s  recordings  regarding  either 
the  symptoms  indicated  or  the  treatment  plan  developed.  Considered  con¬ 
jointly,  the  findings  were  that  nearly  all  of  these  282  DCSs,  97.1  percent, 
evidenced  an  error  of  omission — a  lack  of  a  consultation  with  a  physician. 

5.2  Findings  Related  to  Medical  Effectiveness  of  Care  Delivered 
(Patient  Return  Rates).  Among  the  eight  general  categories  of  illness 
addressed  by  the  AMOSIST  algorithms,  the  overall  proportion  of  AMIC-treated 
patients  who  made  non-directed  returns  to  obtain  additional  care  for  the 
same  or  a  related  illness  as  had  been  treated  than  that  evidenced  among 
GOC-treated  patients. 

5.3  Additional  Findings. 

5.3.1  Overall,  among  all  905  DCSs  of  AMIC-treated  patients  included  in 
the  retrospectice  record  audit,  the  data  were  entered  upon  current  DCSs  for 
83.6  percent  of  the  patients,  upon  outdated  DCSs  for  3.2  percent  of  the 
pateints,  and  upon  SF  600s  for  the  remaining  13.2  percent  of  the  patients. 
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5.3.2  Frequency  of  Occurrence  Among  Diagnosis  Rendered.  Among  all  905 
DCSs  Included  In  Che  recrospectlve  audit,  nearly  40  percent  (20  diagnosis) 

of  Che  52  specified  diagnoses  Included  in  the  present  AMOSIST  Manual  occurred 
two  times  or  less;  nine  of  these  did  not  occur  at  all. 

5.3.3  DCS  Format.  The  format  of  the  DCSs  presently  in  use  in  the 
AMOSIST  Program  does  not  provide  a  clear  and  readily  understandable  record 
of  the  logic  employed  by  AMOSIST  in  arriving  at  a  diagnosis  and  determining 
a  treatment  plan.  Nor  do  the  data  recording  procedures  employed  enable 
AMOSIST  physicians  performing  requested  consultations  to  document  the 
extent  and  nature  of  their  Involvement  in  the  treatment  of  an  AMOSIST' s 
patient. 

6.  CONCLUSIONS. 

6.1  Safety  of  Care. 

6.1.1  As  a  result  of  the  exceptionally  high  error  rate,  the  AMOSIST 
program  is  viewed  to  be  "at  risk"  from  a  medico- legal  perspective. 

6.1.2  The  present  AMOSIST  Manual  is  overburdened  with  instructions 
and  logic  pertaining  to  infrequently  utilized  diagnoses;  a  condition 
which  is  believed  to  be  contributing  to  the  lack  of  its  use  by  AMOSISTs 
in  the  field  and,  consequently,  to  the  high  error  rate  evidenced. 

6.1.3  The  validity  and  efficiency  of  existing  and  envisioned  treat¬ 
ment  DCS  auditing  procedures  is  degraded  by  the  lack  of  current  program- 
prescribed  treatment  DCSs  to  record  the  findings,  treatment  and  disposi¬ 
tion  of  all  AMOSIST-treated  patients. 

6.1.4  The  format  of  the  current  data  collection  sheets  and  the 
procedures  used  by  the  physicians  to  record  the  extent  and  nature  of 
their  impact  upon  the  treatment  of  an  AMOSIST' s  patient  are  incompatible 
with  a  requirement  to  efficiently  and  effectively  perform  manual  audits 
of  treatment  DCSs. 

6.2  Effectiveness  of  Care.  The  AMOSIST  Program  has  the  potential  to 
deliver  care  which  is  not  significantly  different  in  its  effectiveness 
than  that  which  would  be  delivered  by  physicians  treating  a  comparable 
sample  of  patients. 

7.  RECOMMENDATIONS. 

7.1  Principal  Recommendations .  The  principal  recommendations  arising 
from  the  present  report  are  that: 

a.  A  new  series  of  DCSs  and  data  collection  procedures  be 
developed  that  will  clearly  depict  (1)  the  logic  utilized  by  the  AMOSIST 
in  examining  and  treating  their  patients,  and  (2)  the  nature  and  extent 
of  physician  Impact  upon  the  algorithmic  care  delivery  process  which 
results  from  AMOSIST-requested  physician  consultations. 
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APPENDIX  D 


AMOSIST  PROGRAM  FIELD  EVALUATION:  STABILITY  AND  VIABILITY— A 
RE-EVALUATION  OF  PROGRAM  ACCEPTANCE  AND  OPERATIONAL  CHARACTERISTICS 


5.  SUMMARY  OF  FINDINGS. 


5.1  AMIC  Checklist.  The  operations  of  the  AMICs  evaluated  evidenced 
considerable  change  between  Phases  I  and  II  although  there  existed  no 
statistically  significant  change  in  either  a  positive  or  negative  direction. 
The  area  evidencing  the  most  positive  change  was  that  pertaining  to  the 
triage  function.  The  most  negative  changes  were  evidenced  in  the  areas  of 
AMOSIST  supervision  and  auditing  functions.  (The  latter  is  believed  to  be 
due  largely  to  the  use  of  terminology  which  unambiguously  defined  "audit" 

to  mean  a  "formal,"  recorded  audit.)  Aside  from  the  extent  of  change, 
per  se,  the  findings  continued  to  evidence  low  levels  of  functioning  in 
the  critical  areas  of  AMOSIST  supervision,  auditing  procedures,  and 
physician  staffing  practices. 

5.2  Patient  Satisfaction.  Among  the  sites  included  in  the  mailed 
surveys,  there  was  a  statistically  significant  increase  between  Phase  I 
and  Phase  II  findings  regarding  the  level  of  patient  satisfaction 
evidenced.  The  comparison  between  patient  satisfaction  present  at  the 
three  AMICs  and  the  three  General  Outpatient  Clinics  (GOCs)  included  in 
the  Phase  II  evaluation  of  Cost  Effectiveness  and  Physician  Savings 
(Schopper,  1978b)  showed  patients  treated  at  AMICs  were  significantly  more 
satisfied  with  the  treatment  received  than  those  treated  at  GOCs. 

5.3  Staff  Satisfaction 


5.3.1  AMOSISTs.  The  interphase  comparisons  showed  AMOSISTs  in  Phase 
II  to  be  significantly  less  satisfied  with  their  work  than  were  AMOSISTs 
in  Phase  I.  Additionally,  there  were  significantly  more  91Bs  and  signif¬ 
icantly  fewer  91Cs  in  Phase  II  that  were  in  Phase  I.  Since  the  91B  in 
Phase  II  was  also  significantly  less  satisfied  with  his  job  than  his  91C 
counterpart  (a  finding  consisted  with  Phase  I  results),  the  interphase 
decrement  was  hypothesized  to  be  due  to  the  prevalence  of  the  91B  in  the 
Phase  II  sample.  In  contrast  to  the  Phase  I  finding  of  greater  job 
satisfaction  among  AMOSISTs  than  among  non-AMOSISTs,  the  job  satisfaction 
of  Phase  II  AMOSISTs  was  found  to  be  significantly  different  from  their 
non-AMOSIST  counterparts.  As  in  Phase  I,  however,  AMOSISTs  in  training 
status  at  the  time  of  the  Phase  II  evaluation  were  found  to  be  more 
satisfied  than  their  more  experienced  co-workers.  A  similar  interphase 
consistency  was  found  regarding  the  RAP  versus  non-RAP  evaluation. 
AMOSISTs  serving  at  AMICs  wherein  roster-assigned-physicians  provided  a 
significant  portion  of  the  physician  man-hours  supplied  to  the  clinic 
were  significantly  less  satisfied  with  their  work  than  those  serving  at 
AMICs  staffed  by  regularly  assigned  physicians. 

5.3.2  AMOSIST  Physicians.  AMOSIST  Physicians  responding  in  Phase  II 
were  significantly  less  satisfied  with  their  work  than  were  those  who 
responded  in  Phase  I.  They  were  not,  however,  significantly  different 
from  Phase  II  physicians  serving  in  other  outpatient  clinics.  Their 
evaluations  of  AMOSISTs  and  the  AMOSIST  Program  evidenced  a  significant 
decrease  between  Phases  I  and  II.  Issues  pertaining  to  the  AMOSISTs 
competence  were  prominent,  and  it  is  believed  that  the  decrements  shown 
were  due,  in  large  measure,  to  the  substantially  increased  proportion  of 
the  less  formally  educated  91B  (vis-a-vis  the  91C)  working  in  the 
AMOSIST  Program  at  the  time  of  the  Phase  II  evaluation. 
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5.3.3  Non-AMIC  Staff  Perceptions.  Statistically  significant  inter¬ 
phase  changes  were  observed  in  the  comments  rendered  regarding  the  Program. 
In  consonance  with  the  perceptions  of  the  AMOSIST  MD  cited  above,  these 
changes  evidenced  a  lack  of  confidence  in  the  competence  of  AMOSIST  and, 
too,  are  believed  to  reflect  the  increase  in  the  relative  percentage  of 
91Bs  (versus  91Cs)  in  the  AMOSIST  Program  at  the  time  of  the  second  phase 
evaluation. 

6.  CONCLUSIONS. 

6.1  Operational  Characteristics. 

6.1.1  There  occurred  no  overall  statistically  significant  directional 
change  in  the  operational  characteristics  of  the  AMOSIST  Program  between 
Phase  I  and  Phase  II. 

6.2.1  The  program  continues  to  remain  acutely  deficient  as  regards  the 
performance  of  adequate,  documented  audits  of  the  AMOSIST's  performance — 

a  function  which  is  believed  (in  view  of  the  small  amount  of  formal  medical 
education  provided  by  the  AMOSISTs)  to  be  a  critical,  core  requirement  of 
the  program. 

6.1.3  Inadequate  physician  staffing  and  supervision  practices  continue 
to  exist.  Checklist  data  indicate  that  physicians  who  are  not  regularly 
assigned  to  the  AMIC  continue  to  provide  a  substantial  portion  of  the 
physician  man-hours  provided  to  some  clinics  (a  practice  which,  according 
to  AMOSISTs  responses  on  the  job  satisfaction  questionnaire,  is  deleterious 
to  their  morale  and  their  stated  likelihood  of  remaining  in  the  Army) . 

Data  from  both  the  AMIC  Checklist  and  from  the  AMOSISTs'  responses  to  the 
questionnaire  addressing  their  work  satisfaction  Indicate  that  physician 
supervision  is  not  available  to  AMOSISTs  at  some  sites  during  the  entirety 
of  their  working  hours. 

6.1.4  Overall,  the  operational  characteristics  of  the  program  are 
such  that  it  is  considered  to  remain  "at  risk"  from  a  medico-legal 
perspective. 

6.2  Program  Acceptance. 

6.2.1  Patient  Satisfaction.  The  manner  in  which  medical  care  is 
delivered  by  AMOSIST  Program  care  providers  is  acceptable  to  the 
recipient  of  such  care. 

6.2.2  Staff  Satisfaction.  As  the  apparent  result  of  the  replacement 
of  the  better-educated  91C  AMOSIST  with  the  lesser-educated  91B 
AMOSIST  (a)  the  job  satisfaction  of  both  AMOSIST  physicians  and  AMOSIST 
themselves  decreased  between  Phases  I  and  II,  and  (b)  the  rating  given 

by  both  the  AMOSIST  physicians  and  the  hospital  staff  in  general  regarding 
the  perceived  competence  of  AMOSISTs  decreased  during  the  interim  between 
Phase  I  and  Phase  II. 
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RECOMMENDATION 


The  findings  and  conclusions  rendered  In  the  present  report 
are  in  consonance  with,  and  reinforce  the  findings  and  conclusions  of 
the  Initial  report  of  this  study  (Schopper,  1978a) .  As  a  result  of 
the  present  replication,  the  recommendations  of  that  report  remain 
relevant  and  receive  added  emphasis.  The  exception  pertains  to  the 
issue  of  physician  staffing.  Rather  than  recommencing  at  least  a  one- 
half  time  regularly  assigned  physician  to  each  AMIC  as  was  initially 
advocated  (para  6.5,  Schopper,  1978a)  it  Is  now  recommended  that  (a) 
each  AMOSIST-staf fed  clinic  have  a  minimum  of  one  regularly  assigned 
physician  present  in  the  clinic  during  the  entirety  of  its  hours  of 
operation,  and  (b)  the  roster  assignment  of  physicians  not  be  permitted 
as  principal  physician  supervisor  to  AMOSISTs.* 


*No  recommendation  is  made  for  remedial  action  concerning  the  decrement 
observed  between  Phases  I  and  II  regarding  the  perceived  competence  of 
AMOSISTs.  The  perceived  degradatlon-of-competence  was  cited  by  both 
AMQSIST  Physicians  and  non-AMIC  hospital  staff.  This  is  concomitant 
with  the  withdrawal  of  9 1C  from  the  program  and  appears  to  be  an  ex¬ 
pression  of  anticipation  rather  than  of  actual  observation.  The  other 
observer  group,  the  patients,  did  not  share  their  loss  of  faith  but 
actually  expressed  a  slight  Increase  in  patient  satisfaction.  It  is 
also  the  opinion  of  the  study  workers  that  Implementation  of  a  better 
controlled  and  operated  program  will  do  more  to  Increase  confidence  in 
the  program  than  changing  the  admission  requirements  for  the  AMOSIST. 
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